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DETAILED ACTION 

Response to Amendment 

1 . Applicant's response to the last Office Action, filed September 1 st 2005 
has been entered and made of record. 

2. Applicant has amended Claims 1 , 6, 1 1 , 1 4, 1 9, 31 , 33, 38, 43 and 46 are 
amended. Claims 17 and 49 are canceled. New Claims 51-56 have been added. 

3. Applicants remarks have been entered and considered but are 
notcompletely persuasive for at least the following reasons: 

4. With regard to amended independent claims 1 and 33, Applicant has 
amended the claims to include the limitation from canceled claims 17 and 49. Applicant 
argues that the reference to Benati does not disclose the limitation now added to the 
independent claims of 1 and 33. Applicant points to the passage in Benati of column 8, 
line 62-column 9, line 13. Applicant points to the line that "upon correction, a reduction 
of this high luminance value is required." Examiner points out that the limitation claimed 
reads as follows "wherein said first predefined color has a first brightness intensity, and 
wherein said image modifying module is further effective for changing the pixels of said 
digital image bordering said first image pixels to a second common color having a 
second brightness intensity higher than first brightness intensity." This reads plainly on 
the effect generated by Benati. Benati discloses that "the luminance channel is 
multiplied by a factor of 0.35 in order to reduce the lightness of the pixel's neutral value." 
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This is in reference to the actual red eye pixels, which are decreased in brightness by a 
factor of 0.35 giving them an effective comparative brightness value of 0.65. Benati 
then states (column 9, lines 9-12) that the border pixels surrounding the red eye pixels 
are "only darkened by a 0.15 factor." This would give the bordering pixels an effective 
comparative brightness value of 0.85 which is effectively brighter than the first 
brightness intensity. Furthermore with the previous presented discussion regarding the 
canceled claims 17 and 49, it would have been obvious to modify the border pixels as 
desired to achieve amore natural looking picture as taught by Benati. Therefore the 
reference of Benati still reads on the added limitation in the amended independent 
claims 1 and 33. The subject matter added to the independent claims does not 
distinguish the present invention from the prior art and is therefore insufficient for 
allowance. Therefore the rejection of claims 1 and 33 and their subsequent dependent 
claims is maintained and accordingly made final. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, nnore than one year prior to the date of application for patent In the United 
States. 

Claims 54-56 are rejected under 35 U.S.C. 102(b) as being anticipated by U.S. 
Patent 5,990,973 to Sakamoto. 
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With regard to claim 54, Sakomoto discloses a method of redeye reduction for a 
digital image said method comprising: 

Cycling through the pixels of said digital image to identify a first red eye region 
(column 3, lines 12-20). 

Responding to the identifying of said first red eye region by initiating a search for 
a second red eye region (column 3, lines 20-35); and 

Modifying the color of the identified first and second red eye regions (column 3, 
lines 40-45). 

With regard to claim 55, Sakamoto discloses the method of claim 54, wherein the 
search for said second red eye region includes using a measure of a characteristic 
parameter of the identified first red eye region as a reference parameter to locate said 
second red eye region (column 3, lines 20-30). The color information from the first red- 
eye area is used to search for the second red-eye region. 

With regard to claim 56, Sakomoto discloses a method of redeye reduction for a 
digital image, said method comprising: 

Identifying a first red eye region (column 3, lines 12-20); 

Identifying a second red eye region (column 3, lines 20-30); 

Dilating the size of the identified first and second red eye regions (column 3, lines 
35-45 see area expansion means, interpreted as dilating); and 
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Modifying the color of the dilated first and second red eye regions (column 3, lines 40- 
46). 

Rejections Under U.S.C. § 103(a) 

5 The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 

6. Claims 1-4, 17, 18, 33-36, 49 and 50 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Patti et al. ("Automatic Digital Redeye Reduction", IEEE 1998), 
in view of Acker et al. (U.S. Patent 6,009,209), in further view of Benati et al. (U.S. 
Patent 5,748,764). 

7. The following is in regard to Claim 33. Patti et al. disclose a redeye reduction 
method for reducing the effects of redeye in a digital image that includes at least one 
redeye region. This method comprises: 

a. The step of generating a binary mask (matrix) for identifying candidate 
redeye pixels. See the section Create Mask Algorithm in Patti et alV The 



1 



For convenience, the equations found on the second page of Patti et al. will be referred to as equations (2. 1), (2,2) and (2.3), respectively. 
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algorithm to create this mask involves converting the pixels of the image to 
a non-standard luminance-chrominance representation (color-based 
parameter) and thresholding these converted pixels into candidate pixels 
and non-candidate pixels. Converted pixels having a value less than this 
threshold are assigned a logic value of zero {bfp =o), indicating that they 

are non-candidate pixels, and converted pixels having a value greater 
than or equal to this threshold are assigned a logic value of one {bf^ * 

indicating that they are candidate pixels. See Patti et al. equation (2.2). 

b. A segmentation step. Though Patti et al do not explicitly indicate this step, 
it should be apparent that the binary mask effectively segments the image 
into multiple groups of contiguous candidate 

pixels - i.e. cohesive segments (henceforth, referred to as segments). 
These groups are identified as isolated regions of the binary mask having 
bjp =1. See Patti et al. Figs. 2-4. Using the mask, the pupil (i.e. first target 

cohesive group/first redeye region) is then identified (second stage of flow 
diagram depicted in Patti et al. Fig. 2). 

c. An image-modifying (correction) step. Since elements of the binary mask 
have a one-to-one correspondence with the converted image pixels (i.e. 
bij H> Cr^j ), identification of the image pixels associated with the pupil 
follows trivially from identifying the pupil within the binary mask, as in step 
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(b). The image-modifying step involves replacing these image pixels with 
pixels of a common predetermined color. See last paragraph of Section 4 
of Patti et al. 

Note, however, that Patti et al. does not show or suggest identifying the first target 
segment (i.e. pupil region), among the plurality of segments, as being the segment 
having the candidate pixel with the color-based parameter of highest value among all of 
said plurality of segments. 

8. Acker et al. show, within the context of an automatic redeye removal/reduction 
method, that "automatic identification of the red eye region is based on characteristics 
common to all red eye artifacts - e.g., its shape tends to be compact and circular, with 
high color saturation and a distinct brightness maxima. These characteristics can be 
combined directly into a unified figure of merit that is optimized by searching for a 
roughly circular shape, surrounding the single point marked by the user, that maximizes 
these red eye characteristics" (Acker et al. column 6, lines 13-20). Thus, a region or 
segment of the image, having high color saturation and distinct brightness maxima, 
would be a likely candidate for the redeye (pupil) region. Such properties would 
correspond to a maximum in the aforementioned color-based parameter. This implies 
that searching for a segment or region of the image, which is preferably circular and 
maximizes color saturation and brightness (and, correspondingly, the color-based 
parameter), represents a viable method for locating or identifying the redeye (pupil) 
region. 
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9. It would be a relatively simple undertaking for one of ordinary skill in the art to 
incorporate this teaching into the method of Patti et al. by either identifying and/or 
locating the pupil region by searching, for the segment or region of the image that 
maximizes the color-based parameter, in lieu of the algorithm for pupil location taught 
by Patti et al., or by supplementing (say, by preceding) that algorithm with a search for 
the segment or region of the image that maximizes the color-based parameter. In either 
case, such a modification would represent an improvement over the method of Patti et 
al., in terms of efficiency, because the search space is effectively limited to the 
segments indicated by the binary mask, as opposed to an exhaustive search of the 
entire image plane. Given this advantage and the relative ease of such a modification, it 
would have been obvious to one of ordinary skill in the art, at the time of the applicant's 
claimed invention, to incorporate the teachings of Acker et al. into the method of Patti et 
al. by either identifying and/or locating the pupil region by searching for the segment or 
region of the image that maximizes the color-based parameter, in lieu of the algorithm 
for pupil location taught by Patti et al., or by supplementing (say, by preceding) that 
algorithm with a search for the segment or region of the image that maximizes the color- 
based parameter. In doing so, one would obtain a method of redeye reduction that 
satisfies all limitations of claim 33. 

10. The following is in regard to the subject matter of canceled claims 17 and 49 now 
included in the independent claims 1 and 33. As shown above, the teachings of Patti et 
al. can be combined with those of Acker et al. in such a way as to satisfy all the 
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limitations of claim 33. The image-modifying step (see discussion above relating to 
claim 33) of Patti et al/s redeye reduction method involves replacing the color of pixels 
within the pupil region with a predetermined color having a first brightness intensity and 
having a first chrominance. See last paragraph of Section 4 in Patti et al. The image- 
modifying step, as taught by Patti et al., changes the color of the pixels bordering the 
pupil region to the same predetermined color as the pixels within the pupil region. 
Therefore, neither Acker et al. nor Patti et al. teach changing the pixels of the digital 
image bordering the pupil region to a common color having a second brightness 
intensity higher than said first brightness intensity. 

1 1 . Benati et al. disclose an automated method for the detection and correction of 
eye color defects due to flash illumination (with an emphasis on redeye effects). In 
Benati et al.'s method, border correction is performed on pixels neighboring at least one 
eye color defect pixel (i.e. a pixel in the pupil region). The colors of these neighboring 
pixels are changed so that they have the same chrominance as the pixels within the 
pupil region, but a higher luminance. See Benati et al. column 8, line 62-65 and column 
9, lines 9-12. According to Benati et al. (Benati et al. column 2, lines 40-44), pixels on 
the border of the pupil region are corrected in this manner to create a more natural 
appearing correction. 

12. Benati discloses that "the luminance channel is multiplied by a factor of 0.35 in 
order to reduce the lightness of the pixel's neutral value." This is in reference to the 
actual red eye pixels, which are decreased in brightness by a factor of 0.35 giving them 
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an effective comparative brightness value of 0.65. Benati then states (column 9, lines 
9-12) that the border pixels surrounding the red eye pixels are "only darkened by a 0.15 
factor." This would give the bordering pixels an effective comparative brightness value 
of 0.85 which is effectively brighter than the first brightness intensity. Furthermore with 
the previous presented discussion regarding the canceled claims 17 and 49, it would 
have been obvious to modify the border pixels as desired to achieve amore natural 
looking picture as taught by Benati. Therefore the reference of Benati still reads on the 
added limitation in the amended independent claims 1 and 33. 

13. Given the fact that the redeye reduction method, obtained by combining the 
teachings of Patti et al. and Acker et al. in the manner discussed above, identifies 
border pixels and attempts to correct them, it would be straightforward for one of 
ordinary skill in the art to simply modify this correction so that border pixels are treated 
in the manner suggested by Benati et al. According to Benati et al. (Benati et al. column 
2, lines 40-44), pixels on the border of the pupil region are corrected in this manner to 
advantageously create a more natural appearing correction. Taking this into account, it 
would have been obvious to one of ordinary skill in the art, at the time of the applicant's 
claimed invention, to modify the image modifying/correction step of the redeye reduction 
method, obtained by combining the teachings of Patti et al. and Acker et al. in the 
manner discussed above, so that border pixels are treated in the manner suggested by 
Benati et al. In doing so, one would obtain a redeye reduction method that conforms to 
the limitations of claim 49. 
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14. The following is in regard to Claims 17. This claim recites substantially the same 
limitations as claim 49. Therefore, with regard to claim 17, remarks analogous to those 
presented above relating to claim 49 are applicable. 

15. See also the discussion above in regard to applicant's remarks. The addition of 
this subject matter does not distinguish the present invention over the prior art found 
and is therefore insufficient for allowance. 



16. The following is in regard to Claim 34-35. As shown above, the teachings of Patti 
et al. can be combined with those of Acker et al. in such a way as to satisfy all the 
limitations of claim 33. The redeye reduction method of Patti et al. further includes the 
erosion of candidate segments. See Patti et al. equation (2.3). The erosion step refines 
the aforementioned binary mask and occurs between step (a) - i.e. creating a 
preliminary binary mask according to Patti et al. equation (2.2)) - step (b) - i.e. 
segmentation and pupil location. See the second column on page 2 of Patti et al. Patti 
et al. also perform this erosion twice. See the last paragraph on page 2 of Patti et al. 
Note that executing erosion according to equation (2.3) will erode pixels with less than 
two neighbors. It would have been obvious to one of ordinary skill in the art to initially 
erode pixels with less than three neighbors, as opposed to two, initially, since this would 
result in a more pronounced elimination of extraneous pixels around the edges of the 
candidate segments as well as small, spurious segments unlikely to represent the pupil. 
The number of neighbors in each iteration of the erosion and even the number of 
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iterations are clearly parameters of Patti et al/s redeye reduction method that can be 
readily adjusted depending on the effectiveness of the subsequent segmentation. In this 
way, the teachings of Patti et al. and Acker et al., when combined in the manner 
described above, can be easily modified to conform to claims 34-35. 

1 7. The following is in regard to Claim 36. As shown above, the teachings of Patti et 
al. can be combined with those of Acker et al. in such a way as to satisfy all the 
limitations of claim 34. Again referring to Patti et al. equation (2.3), notice eroded pixels 
are set to 0, the second logic level. In this way, the teachings of Patti et al. and Acker et 
al. can be combined in such a way as to satisfy all limitations of claim 36. 

1 8. The following is in regard to Claim 50. As shown above, the teachings of Patti et 
al. can be combined with those of Acker et al. in such a way as to satisfy all the 
limitations of claim 33. Patti et al.'s method of redeye reduction begins by transforming 
the 1/3 power of the input RGB image to an YCrCb representation and then defining a 
first threshold TDCravgQCrmax - Crmin for use in the subsequent thresholding operation 
(step (a) above). See the last paragraph on page 1 of Patti et al. and Patti et al. 
equation (2.1). Therefore, the teachings of Patti et al. and Acker et al., when combined 
in the manner described above, address all limitations of claim 50. 

1 9. The following is in regard to Claims 1-4 and 18. These claims recite substantially 
the same limitations as claims 33-36 and 50, respectively. Therefore, with regard to 
claims 1-4 and 18, remarks analogous to those presented above relating to claims 33- 
36 and 50 are, respectively, applicable. 
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20. Claim 5 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Patti et al., Acker et al., and Benati et al. as applied to claims 1 and 
33 above, in further view of Kinjo et al. 

21 . The following is in regard to Claim 37. As shown above, the teachings of Patti et 
al. can be combined with those of Acker et al. and Benati et al. in such a way as to 
satisfy all the limitations of claim 33. Patti et al. and Acker et al. and Benati do not teach 
a redeye reduction method wherein the component-connecting step (see the discussion 
above regarding claim 33) further includes generating statistical data for each of said 
plurality of cohesive groups including a high parameter value indicating the highest 
color-based parameter value of its corresponding candidate pixels, an average 
parameter value indicating the average color-based parameter value of its 
corresponding candidate pixels, and a size parameter value indicating its size. 

22. Kinjo et al. discloses an image processing method for the reduction of redeye 
effects in a digital image. This image processing method includes a segmentation 
(component-connecting) step, wherein, for each segment, at least one of five marks is 
determined. These marks consist of a first mark indicating the roundness of the 
segment, a second mark indicating the position of the segment, a third mark indicating 
the surface area of the segment, a fourth mark for expressing the degree of poorness of 
color tone of the segment, and a fifth mark (the fifth mark is not relevant to this 
discussion). See, for example, Kinjo et al. column 5 lines 15-44. Note that the first 
through third marks, together, provide a measure of the size of the segment. Kinjo et al. 
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suggests that the fourth mark can comprise at least one of an average value and 
maximum value of at lease one of the hue, saturation, and lightness (which together 
define the color of) the segment. See Kinjo et al. column 5 lines 15-44. In this way, the 
first through third marks represent a size parameter, similar to that of the applicant's 
claimed method, and the fourth mark can provide a high parameter value and an 
average parameter value, similar to those of the applicant's claimed method. 
23. Deriving such marks for the segments, determined by the component-connecting 
step of the redeye-reduction method obtained by combining the teachings of Patti et al. 
and Acker et al. in the manner discussed above, would be a simple undertaking for one 
of ordinary skill in the art, given the teachings of Kinjo et al. The information contained in 
these marks would provide an effective, alternative means to evaluate the size and 
shape of the segments obtained by the segmentation/component-connecting step of the 
redeye-reduction method obtained by combining the teachings of Patti et al. and Acker 
et al. This is an attractive alternative for locating the pupil region because using the 
marks of Kinjo et al., in the manner suggested by Kinjo et al., reduces the need for the 
exhaustive search scheme suggested by Patti et al. Similarly, the marks indicating the 
color properties of the segment can facilitate the search for pixels surrounding the 
periphery of the located pupil region, as taught by Patti et al. Given the facility of the 
information encompassed by Kinjo et al.'s marks and the relative ease of incorporating 
these marks into the method obtained by the combined teachings of Patti et al. and 
Acker et al., it would have been obvious to one of ordinary skill in the art, at the time of 
the applicant's claimed invention, to derive marks, as taught by Kinjo et al., containing 
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information regarding size, shape, location, maximum/minimum color, and average 
color, for each of the segments determined by the component-connecting/segmentation 
step of the method obtained by the combining the teachings of Patti et al. and Acker et 
al., in the manner discussed above. In doing so, one would obtain a method that 
conforms to the limitations put forth in claim 37. 

24. The following is in regard to Claims 5. This claim recites substantially the same 
limitations as claim 37. Therefore, with regard to claim 5, remarks analogous to those 
presented above relating to claim 37 are applicable. 

New Claims 

Claim 51 is found to be allowable for the same reasons as claims 6-16 and 38-46 
as discussed below. 

Claims 52 and 53 are found to be allowable because of the inclusion of the 
second thresholding step and the boundary adjusting module previously indicated as 
allowable subject matter with regard to claim 28. 

Claims 54-56 are rejected under 35 U.S.C. 102(b) as being anticipated by U.S. 
Patent 5,990,973 to Sakamoto as discussed above. 
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, Allowable Subject Matter 

Claims 19-32 were allowed in the previous office action. 

Claims 6-16, 19-32 and 38-48 and 51-53 are allowable as amended to include 
allowable subject matter indicated in the previous office action or as indicated allowable 
in the last office action. 

25. New claim 51 is allowable for the same reasons as indicated in the first 
office action in regard to the allowable subject matter discussed in claims 6-16 and 38- 
48. None of the prior art cited or found teaches or fairly suggests performing a second 
target-determination according to these claims, in addition to the first target- 
determination. Specifically, prior art does not show searching for a regions, 
corresponding to a potential second pupil, having an average size and/or color that lies 
within a range dependant on the average size and/or color of the first pupil. It should be 
noted, however, that the effect of the pupil location algorithm in the method of Patti et al. 
is that segments having an average size that is not optimally correspondent to that of a 
pupil are filtered out. This, however, cannot be construed as fully satisfying the 
limitations of claims 6 or 38. 

26. The following is in regard to Claims 7-8 and 39-40, As mentioned above, the 
encountered prior art failed to teach redeye reduction systems or methods that conform 
to the limitations of claim 6 or 38. However, it should be understood that the selection of 
the bounds of size and/or color parameter ranges that are dependant on the first pupil's 
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size and/or color parameters can be made arbitrarily and, therefore, would have been 
obvious to one of ordinary skill in the art. Therefore, while the limitations of claims 7-8 
and 39-40, relating to the numerical bounds of these ranges, are, in and of themselves, 
not allowable, these claims are allowable by virtue of their dependence on their 
allowable claims 6 and 38, respectively. 

27. The following is in regard to Claims 9 and 41, As mentioned above, the 
encountered prior art failed to teach redeye reduction systems or methods that conform 
to the limitations of claim 6 or 38. Patti et al., however, do locate the region 
corresponding to a pupil via a block matching algorithm that attempts to find the 
smallest s x s block in the input mask with the maximum number of white pixels (see 
Patti et al. equation (2)) - that is, the block with the highest boundary square population. 
Thus, pupil location is accomplished in essentially the same way as the applicant's 
second-target determination. In this way, the identification of the second target cohesive 
group, as proposed by the applicant, is not unique. However, the applicant's initial 
determination of the first pupil by searching for the segment with the highest color- 
content (e.g. saturation), and the subsequent usage (prior to the said determination of 
the boundary square population) of the size and/or color information of the found first 
pupil to eliminate segments within a specified range of the first pupil's color and/or size, 
is not shown in prior redeye reduction methods. 

28. The following is in regard to Claims 10 and 42. As mentioned above, the 
encountered prior art failed to teach redeye reduction systems or methods that conform 
to the limitations of claim 9 or 41 . Notice that claims 10 and 41 claim a modification of 
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the second image pixels that is analogous to the modification of the first image pixels. . 
Therefore, while the limitations of claims 10 and 42, relating to the modification of the 
second image pixels, are, in and of themselves, not allowable, claims 10 and 42 are 
allowable by virtue of their dependence on their allowable claims 9 and 41 , respectively. 

29. The following is in regard to Claims 1 1-14 and 43-46. These claims are similar to 
claims 25-27. Please refer to the discussion below, with regard to claims 25-28, for the 
justification of allowing these claims. 

30. The following is in regard to Claims 15-16 and 47-48. These claims are allowable 
for essentially the same reasons as claims 12-13 and 44-45, respectively. See the 
remarks above relating to those claims. 



Conclusion 

Applicant's amendment necessitated the new grounds of rejction presented in 
the Office Action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Wes Tucker whose telephone number is 571-272-7427. 
The examiner can normally be reached on 9AM-5PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 571-272-7414. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




Wes Tucker 
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